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LONG-TERM  GOALS 

Tracking  and  predicting  the  fine  scale  sea  ice  motion  will  be  particularly  critical  to  safe  Navy 
operations  in  the  Arctic  Ocean.  It  is  also  very  important  to  monitor  and  track  the  size,  shape,  and 
location  of  sea  ice  in  near  real-time  (within  several  hours  of  data  acquisition),  and  predict  the 
motion  of  sea  ice  for  ice  hazard  forecasts  in  a  finer  scale  than  is  currently  possible.  The  log-term 
goal  of  this  project  is  to  explore  new  algorithms  for  the  purpose  of  constructing  and  analyzing 
super-resolution  images  from  multiple  satellite  sensors  for  automatic  tracking  and  prediction  of 
fine  scale  sea  ice  motion  in  Arctic  Ocean. 

OBJECTIVES 

The  objective  of  our  proposal  is  to:  1)  construct  super-resolution  images  from  satellite  sensors 
which  will  significantly  improve  the  tracking  of  sea  ice  motion  from  a  coarse  scale  to  a  finer 
scale;  and  2)  implement  data  fusion  methods  to  combine  data  from  passive  microwave  and  other 
satellite  sensors  to  produce  more  accurate  tracking  and  prediction  of  sea  ice  motion. 
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APPROACH 


We  propose  to  explore  new  algorithms  to  construct  and  analyze  super-resolution  images  from 
multiple  satellite  sensors  for  automatic  tracking  and  predicting  fine  scale  motion  of  Arctic  sea 
ice.  Our  research  efforts  will  focus  on  the  following  main  tasks: 

a)  Constructing  super-resolution  images  from  multiple  satellite  sensors  (Dr.  Yingli  Tian 
and  students ):  There  are  many  methods  for  constructing  super-resolution  images  for 
video  surveillance  and  image  retrieval.  We  will  extrapolate  and  adapt  selected  techniques 
to  construct  super-resolution  images  from  satellite  imagery  that  has  a  resolution  at  least  5 
times  higher  than  that  of  the  original  satellite  imagery. 

b)  Tracking  sea  ice  motion  from  multiple  satellite  sensors  and  passive  microwave  sensors 
(Drs.  Tian,  Meier,  and  students ):  We  will  develop  robust  methods  to  automatically  track 
sea  ice  motion  from  passive  microwave  sensors  and  multiple  satellite  sensors  respectively. 

c)  Fusion  of  passive  microwave  sensors  and  multiple  satellite  sensors  to  track  fine  scale 
sea  ice  motion  (Drs.  Tian,  Meier,  and  students):  We  will  develop  new  fusion  algorithms 
to  effectively  integrate  data  from  multiple  sensors  for  sea  ice  motion  monitoring. 

d)  Predicting  the  fine  scale  sea  ice  motions  (Drs.  Tian,  Weiss,  and  students ):  We  will 
develop  prediction  models  to  represent  the  motion  and  changes  of  ice  parcels  by  using 
motion  history  obtained  from  the  sea  ice  tracking. 

WORK  PROGRESS 

The  project  started  from  March  2013.  Currently,  we  are  focusing  on  the  task  of " Constructing 
super-resolution  images  from  multiple  satellite  sensors”.  The  progresses  we  have  accomplished 
are  described  as  following: 

1.  Developing  New  Super  Resolution  Algorithms:  Image  super-resolution  is  to  restore  a  high 
resolution  image  from  one  or  multiple  low  resolution  images  which  is  an  essential  operation  in 
many  applications.  It  is  a  challenging  task  to  restore  sharp  edges  while  still  maintains  rich  and 
natural  texture.  We  are  developing  a  novel  gradient-based  single  image  super-resolution  method 
which  takes  benefit  from  internal  across-scale  similarity  as  well  as  external  gradient  statistics  to 
fully  recover  realistic  texture  information.  The  proposed  algorithm  is  able  to  create  clean  edge 
with  minimal  visual  artifacts  while  constructing  fine  detailed  texture  information  which  respects 
the  input  image.  Our  preliminary  experiment  results  demonstrate  that  the  proposed  method 
achieves  the  state-of-art  results.  Moreover,  our  proposed  algorithm  is  easily  scalable  to  include 
multiple  input  images. 

2.  Investigating  the  satellite  data  for  our  research:  Initial  efforts  have  focused  on  finding 
useful  regions  to  collect  satellite  data.  Passive  microwave  data  is  available  in  all  sea  ice 
conditions  and  provides  complete  daily  coverage.  However,  spatial  resolution  is  coarse  and 
smaller-scale  motions  cannot  be  determined  from  passive  microwave  data.  Other  sensors  are 
more  limited,  but  potentially  provide  more  detailed  data.  Initial  assessments  have  been  made  on 
MODIS  data  in  terms  of  its  suitability.  While  clouds  obscure  substantial  regions  of  the  Arctic  at 
any  given  time,  we  have  found  that  enough  clear  sky  exists  to  obtain  incomplete  but  reasonable 
fields  of  motion  estimates. 
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Data  from  Aqua,  Terra,  and  Suomi  NPP  satellites  were  investigated.  Aqua  and  Terra  are  older 
satellites  that  fly  the  MODIS  instrument.  Suomi  is  a  newer  satellite  which  flies  the  VIIRS 
instrument.  MODIS  and  VIIRS  are  scanning  radiometers.  All  three  satellites  orbit  the  earth  in  98 
degree  retrograde  sun  synchronous  orbits,  which  provide  them  with  views  of  the  Arctic  Ocean  on 
every  orbit.  Between  the  three  satellites,  every  point  on  the  Arctic  Ocean  is  viewed  at  least  three 
times  each  day.  The  swaths  that  MODIS  and  VIIRS  view  are  very  wide  (2330  km  for  MODIS 
and  3040  km  for  VIIRS).  The  near  polar  orbits  of  the  satellites  cause  the  swaths  of  each  satellite 
to  converge  and  greatly  overlap  at  the  poles  on  multiple  orbits.  Therefore,  on  any  one  day  many 
points  on  the  Arctic  Ocean  will  be  viewed  more  than  three  times.  For  any  particular  point,  some 
views  will  be  near  nadir  and  many  of  the  views  will  have  significant  slant  angles.  With  a  98 
degree  inclination,  nadir  during  the  most  northern  reach  of  each  orbit  will  be  at  82  degrees  north. 
As  an  example,  82  degrees  north  is  less  than  1000  km  from  the  North  Pole.  Therefore,  every 
orbit  of  all  three  satellites  views  the  North  Pole,  even  though  it  is  always  far  from  nadir. 

MODIS  detects  light  in  wavelengths  from  405  nm  to  14.385  um.  The  light  detectors  are  divided 
into  36  different  spectral  bands.  The  resolution  of  the  detectors  range  from  pixels  that  are  250 
meters  square  to  pixels  that  are  1000  meters  square.  In  comparison,  VIIRS  has  22  spectral  bands 
with  wavelengths  from  402  nm  to  12.48  um.  Pixel  sizes  range  from  371  meters  by  387  meters  to 
742  meters  by  776  meters.  MODIS  and  VIIRS  imagery  data  are  both  available  online.  Data  is 
divided  into  segments  of  orbits.  Each  segment  is  referred  to  as  a  granule.  Each  granule  represents 
5  minutes  of  flight  time.  Therefore,  a  MODIS  granule  is  approximately  2030  km  long  and  2330 
km  wide,  and  a  VIIRS  granule  is  approximately  2030  km  long  and  3040  km  wide. 

3.  Locating  the  geographic  locations  for  our  research:  In  addition  to  general  issues  of 
coverage,  higher  spatial  resolution  imagery  is  more  unwieldy  to  use  for  large  file  sizes  and  thus  it 
makes  sense  to  at  least  initially  focus  on  one  or  two  key  regions.  We  have  chosen  to  first  look  in 
the  Beaufort  and  Chukchi  sea  region  for  several  reasons.  First,  because  it  encompasses  U.S. 
territorial  waters,  it  is  of  key  interest  to  the  Navy.  Second,  it  is  a  region  of  increasing  economic 
interest  due  to  the  potential  for  extraction  of  natural  resources.  Third,  it  is  a  region  where 
changes  in  the  sea  ice  are  dramatic.  Recent  summers  have  seen  extremely  low  ice,  with  the  edge 
receding  several  hundred  kilometers  from  the  coast.  However,  this  summer  provided  an 
interesting  contrast,  with  late  melt  onset  and  the  edge  staying  much  nearer  to  the  coast.  This  was 
despite  the  fact  that  the  preceding  winter  saw  substantial  sea  ice  dynamics,  with  many  large  leads 
forming  during  February  and  March 

(http://earthobservatory.nasa.gov/IOTD/view.php?id=80752).  Such  variability  has  important 
ramifications  for  forecasting  and  operational  activities  in  the  region. 

Working  plan  for  Oct.  2013  —  Sept.  2014: 

1.  We  will  continue  to  improve  and  refine  the  image  super-resolution  algorithm  and  apply  it  to 
satellite  data. 

2.  We  will  develop  object  tracking  method  to  track  multiple  objects  in  crowded  environments. 
The  algorithm  will  be  applied  to  ice  fragment  tracking. 
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3.  We  will  acquire  satellite  data  at  specific  geo-locations  and  time.  The  data  should  be  annotated 
to  track  specific  ice  fragments  and  analyzed  for  predicting  ice  moving  model.  The  super¬ 
resolution  algorithms  will  be  applied  to  generate  images  with  finer  resolution  than  the  original 
MODIS  and  VIIRS  imagery.  The  motion  of  the  ice  fragments  will  be  tracked  on  the  finer 
resolution  images.  The  ice  fragments  will  be  first  manually  initialized  on  the  first  image  frame 
while  our  research  focusing  on  tracking  method. 

RESULTS 

1.  Preliminary  results  of  super  resolution  algorithm:  The  listed  experimental  results  below 
demonstrate  the  feasibility  of  our  proposed  super-resolution  algorithm  in  large  upscaling  factors. 
Sample  image  used  in  Figure  3  is  very  challenging  due  to  the  extreme  richness  and  complexity  of 
MODIS  sea  ice  data.  However,  our  proposed  method  reconstructs  clean  edges  of  the  ice 
fragments  for  further  tracking  purpose. 


Figure  1:  Super-resolution  result  on  ‘kid’  (4  x).  Left  is  the  input  image;  right  is  the  result  after  the  proposed  algorithm.  Results 
are  better  viewed  in  the  zoom-in  boxes. 


Figure  2:  Super-resolution  result  on  ‘chip’  (8  x).  Left  is  the  input  image;  right  is  the  result  after  the  proposed  algorithm.  Results 
are  better  viewed  in  the  zoom-in  boxes. 

2.  The  passive  microwave  sea  ice  product  distributed  by  NSIDC  has  been  updated  through  2012 
and  will  soon  be  made  public  (pers.  communication,  Donna  Scott,  NSIDC).  This  is  the  first 
update  since  2006  and  will  provide  an  updated  product  to  work  with.  The  ice  motion  software 
has  also  been  obtained,  so  that  our  investigating  team  can  potentially  run  motions  directly.  We 
have  also  determined  useful  locations  and  time  periods  to  acquire  imagery  and  sample  images 
have  been  collected. 
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Figure  3:  Super-resolution  result  on  sample  MODIS  image  (4  x).  Left  is  the  input  image;  middle  is  our  reconstructed  result;  right 
is  the  ground  truth  (GT)  image.  Both  input  image  and  the  GT  image  are  captured  from 
http://rapidfire.sci.gsfc.nasa.gov/imagery/subsets/?subset=Arctic_r02c04.2013070.terra  with  resolutions  of  1km  and  250m 
respectively. 


3.  Locating  the  geographic  locations:  Discussions  with  Navy  Task  Force  Climate  Change  have 
indicated  that  the  Beaufort  Sea  would  be  an  appropriate  initial  area  of  interest  from  a  Navy 
perspective.  The  Beaufort  Sea  is  also  of  great  interest  from  a  purely  scientific  perspective.  In 
early  2013,  storms  in  the  Beaufort  Sea  resulted  in  large  areas  of  open  ocean.  The  area  125  W 
long  to  135  W  long  and  70  N  lat  to  75  N  lat  in  the  time  period  January  2013  through  March  2013 
will  be  used  as  the  initial  Arctic  Ocean  area  on  which  this  project  will  focus. 

In  October  2011  a  NASA  IceBridge  flight  spotted  a  large  crack  in  the  Pine  Island  Glacier  in 
West  Antarctica.  Even  though  the  glacier  is  not  sea  ice  and  it  is  not  in  the  Arctic,  a  great  deal  is 
now  known  about  the  dimensions  of  the  crack  as  it  expanded  over  time  and  eventually  caved  a 
large  iceberg  on  8  July  2013.  MODIS  imagery  is  available  for  this  time  period  and  has  been 
obtained.  There  are  also  finer  resolution  images  of  the  crack.  These  will  be  used  for  ground  truth 
experiments  to  test  and  calibrate  the  super-resolution  algorithms. 

IMPACT/APPLICATIONS 

In  addition  to  ice  tracking,  the  proposed  super  resolution  methods  can  be  used  for  many  other 
applications.  The  strong  collaboration  among  the  three  Pis  (Dr.  Tian  from  CCNY,  Dr.  Meier 
from  NASA,  and  Dr.  Weiss  from  NUWC  enhance  the  long-term  collaboration  among  these 
organizations  and  will  bring  research  opportunities  for  the  students  at  CCNY  to  solve  problems 
in  real  applications. 

RELATED  PROJECTS 

W.  Meier:  A  Lagrangian  to  emerging  dynamics  in  the  marginal  ice  zone,  ONR,  Award  Number 
N000141110977,  B.  Tremblay,  PI  -  this  project  uses  the  passive  microwave  derived  sea  ice 
motions  to  track  parcels  of  ice  to  investigate  changes  in  the  source  and  sink  regions  of  Arctic  sea 
ice  and  to  compare  with  CMIP5  model  estimates. 
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